Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add type hints, fix mypy issues (#198) #228

Merged
merged 15 commits into from
Jul 1, 2024

Conversation

jwhitlock
Copy link
Member

This add mypy to the test matrix, and type hints to all the code, fixing #198. mypy required some code changes, and I've rebased them out into separate commits, which could be added before the big one that hints all the functions.

I've built on top of PR #227, which fixes configuration issues with tox.ini. In that PR, I disabled pypy testing against Django 3.2, which was consistently breaking. I thought these changes fixed that, but they still fail 1 out of 10 times locally, so I'm keeping them disabled.

To run mypy locally:

pip install -e .[tests,typing]

* Specify how to install Django 3.2. These tests are currently running
  against later Django versions.
* Add basepython entries for pypy. This fixes running tox locally.
* Fix the cpython version mapping in [gh-actions]. The github action
  tests for cpython versions are running against the latest Django,
  instead of the set of possible Django versions.
There are a few test failures with pypy and Django 3.2. Since 3.2 is out
of long-term support, drop these from the test matrix.
mypy complains when reading or setting a attribute that is not defined
on the class, such as HttpRequest.csp_nonce. This updates the code to
use getattr and setattr to access these dynamically added attributes and
for Django settings.
Both startswith() and parser.parse_statements take a tuple rather than a
list.
Althought the code `template.render(context)` looked similar, mypy
complained that Django's Template could not take a dict. Rather than
switch on types, refactor `make_context` and `make_template` into
`render`, which hides the typing details between Django templates and
extension templates like Jinja2.
Copy link
Member

@robhudson robhudson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is awesome, thanks @jwhitlock!

@robhudson robhudson merged commit 2bb3e6d into mozilla:main Jul 1, 2024
9 checks passed
Copy link
Contributor

@asottile-sentry asottile-sentry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the py.typed file isn't included in the packaging I think? so these types won't be available to users of the package

@@ -15,12 +17,12 @@
class Nonce:
_instance = None

def __new__(cls, *args, **kwargs):
def __new__(cls: Type["Nonce"], *args: Any, **kwargs: Any) -> "Nonce":
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the way to write this is:

def __new__(cls, *args: Any, **kwargs: Any) -> Self:

(Self is importable from typing_extensions for the pythons supported here)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this seems to work. Do you want to submit the PR?

Copy link
Member Author

@jwhitlock jwhitlock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @asottile-sentry. Do you want to submit a PR?

the py.typed file isn't included in the packaging I think? so these types won't be available to users of the package

I was worried about this too, but I think no further config is needed. When I follow the deploy steps in the GitHub action:

pip install build
python -m build -v .

I see the py.typed file getting added to the package data:

...
* Building sdist...
...
copying csp/models.py -> django_csp-4.0b1/csp
copying csp/py.typed -> django_csp-4.0b1/csp
copying csp/utils.py -> django_csp-4.0b1/csp
...
* Building wheel...
...
copying build/lib/csp/py.typed -> build/bdist.macosx-14.4-arm64/wheel/csp
...
adding 'csp/py.typed'
...

@@ -15,12 +17,12 @@
class Nonce:
_instance = None

def __new__(cls, *args, **kwargs):
def __new__(cls: Type["Nonce"], *args: Any, **kwargs: Any) -> "Nonce":
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this seems to work. Do you want to submit the PR?

@asottile-sentry
Copy link
Contributor

Thanks @asottile-sentry. Do you want to submit a PR?

the py.typed file isn't included in the packaging I think? so these types won't be available to users of the package

I was worried about this too, but I think no further config is needed. When I follow the deploy steps in the GitHub action:

pip install build
python -m build -v .

I see the py.typed file getting added to the package data:

...
* Building sdist...
...
copying csp/models.py -> django_csp-4.0b1/csp
copying csp/py.typed -> django_csp-4.0b1/csp
copying csp/utils.py -> django_csp-4.0b1/csp
...
* Building wheel...
...
copying build/lib/csp/py.typed -> build/bdist.macosx-14.4-arm64/wheel/csp
...
adding 'csp/py.typed'
...

the automatic inclusion of py.typed needs at least setuptools>=69: https://github.com/pypa/setuptools/blob/e9f0be98ea4faaba4a7b2d07ba994a81fde8f42f/NEWS.rst#v6900

@jwhitlock
Copy link
Member Author

the automatic inclusion of py.typed needs at least setuptools>=69: https://github.com/pypa/setuptools/blob/e9f0be98ea4faaba4a7b2d07ba994a81fde8f42f/NEWS.rst#v6900

I think we're good then, build installs setuptools>=61.2 so should pick up the latest

https://github.com/mozilla/django-csp/actions/runs/9589504359/job/26443444411#step:6:14

@asottile-sentry
Copy link
Contributor

it relies on an experimental feature and it breaks in environments which don't have setuptools 69+ available -- such as building from source

either the >= should be adjusted (limits where it can be built from source) or it should use the proper package_data stanza to support older setuptools

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants